{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](https://github.com/callysto/callysto-sample-notebooks/blob/master/notebooks/images/Callysto_Notebook-Banner_Top_06.06.18.jpg?raw=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Open Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![open data](http://www.cybera.ca/wp-content/uploads/resizedimage400259-data02-small.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Are Some Datasets Better Than Others? \n",
    "\n",
    "Objectively speaking, yes. Whatever question you're looking to answer, not all open data are easy to work with! \n",
    "\n",
    "### What to look for with open data \n",
    "\n",
    "1. If the data is in multiple files: is it consistently labeled, or easy to relate the quantities between files? \n",
    "![file labels](http://www.cybera.ca/wp-content/uploads/2018/08/Open-data-notebook-label-example.png)\n",
    "1. Is the data in a format that's easy to load on your computer (such as `csv`, `txt`, `json`), or open in Excel?\n",
    "     * A nicely formatted website HTML table works well!  \n",
    " ![html table](https://i.stack.imgur.com/TXh8D.png)\n",
    "\n",
    "1. If you're looking at spacial data: are the GPS coordinates already included (where relevant)?   \n",
    " ![gps coordinates](https://blog.batchgeo.com/wp-content/uploads/2017/01/wikipedia-latitude-longitude.jpg)\n",
    "\n",
    "1. Is the dataset complete? \n",
    "    * If values are missing, is there a good reason for that, and are they clearly labelled as missing?\n",
    "![missing values](https://d2vlcm61l7u1fs.cloudfront.net/media%2Fa06%2Fa06ec409-9969-49bc-ba79-ff006fb92b21%2FphpokoeJU.png)\n",
    "1. Is the data regularly maintained?\n",
    "    * There should be some mention on the download page if it is, or there should at least be a time stamp of when it was last updated.\n",
    " ![timestamp](http://www.cybera.ca/wp-content/uploads/2018/08/whitehorse-data.jpg)\n",
    " \n",
    "\n",
    "\n",
    "### Reliable sources\n",
    "Some online data sets are intentionally misleading or simply incorrect.\n",
    "\n",
    "Much like any other information source, open data is also affected by any biases of its curator. It is important to try and find unbiased data sources, or at least multiple sources of data that have alternate biases. \n",
    "\n",
    "For example, if you're looking to study the heath affects of tobacco, a dataset provided by a study funded by a tobacco company may have different results than an independent study. Be aware of your data's origins before you draw any major conclusions from it! \n",
    "\n",
    "\n",
    "\n",
    "\n",
    "---\n",
    "## What Kind of Open Data Sets Are There, and Where Can I Find Them?\n",
    "\n",
    "There are open data sets available on almost any subject you can think of! A good place to start is with government or academic sites. \n",
    "\n",
    "Here are a few open data portals to get you started:\n",
    "\n",
    "\n",
    "### Government Data\n",
    "#### Canadian Data\n",
    "1. [Statistics Canada](https://www150.statcan.gc.ca/n1/en/type/data?MM=1)\n",
    "1. [Canada Open Data](https://open.canada.ca/en) (Contains many data sets for provinces or territories without their own open data portal)\n",
    "1. [Alberta Open Data](https://open.alberta.ca/opendata)\n",
    "1. [BC Open Data](https://data.gov.bc.ca/)\n",
    "1. [Saskatchewan Open Data](http://www.opendatask.ca/)\n",
    "1. [Northwest Territories Open Data](https://www.opennwt.ca/)\n",
    "1. [Ontario Open Data](https://www.ontario.ca/search/data-catalogue)\n",
    "1. [Quebec Open Data](http://donnees.ville.quebec.qc.ca/catalogue.aspx) (French only)\n",
    "1. [Nova Scotia Open Data](https://data.novascotia.ca/)\n",
    "1. [PEI Open Data](https://data.princeedwardisland.ca/)\n",
    "1. [Calgary Open Data Portal](https://data.calgary.ca/)\n",
    "1. [Edmonton Open Data Portal](https://data.edmonton.ca/)\n",
    "1. [Vancouver Open Data Portal](https://vancouver.ca/your-government/open-data-catalogue.aspx)\n",
    "1. [Toronto Open Data Portal](https://www.toronto.ca/city-government/data-research-maps/open-data/)\n",
    "1. [Winnipeg Open Data Portal](https://data.winnipeg.ca/)\n",
    "1. [Whitehorse Open Data](http://data.whitehorse.ca)\n",
    "\n",
    "_Note 1: Many cities offer their own open data portals, and can often be found through a Google search of \"`CITY NAME` open data portal\"_\n",
    "    \n",
    "_Note 2: Open data portals for the provinces and territories are continually launching at the time of writing._\n",
    "\n",
    "#### Other Governments\n",
    "1. [EU Open Data](https://open-data.europa.eu/)\n",
    "1. [USA](https://www.data.gov/)\n",
    "1. [Australia](https://data.gov.au/)\n",
    "1. [NASA](https://open.nasa.gov/open-data/)\n",
    "1. [Russia](http://data.gov.ru/?language=en) (Site is in English, but many of the datasets are in Russian)\n",
    "\n",
    "_Note 3: There are many national open data portals: these are just a few._ \n",
    "\n",
    "    \n",
    "### Data Aggregators  \n",
    "1. [Kaggle](https://www.kaggle.com/datasets)\n",
    "1. [Open Data Soft](https://data.opendatasoft.com/pages/home/)\n",
    "1. [Open Africa](https://africaopendata.org/)\n",
    "1. [List of interesting data sets](https://github.com/awesomedata/awesome-public-datasets)\n",
    "1. [Open Data Network](https://www.opendatanetwork.com/) (Technically speaking, this is a data set search engine)\n",
    "1. [Google Public Data](https://www.google.com/publicdata/directory)\n",
    "\n",
    "---\n",
    "\n",
    "This is by no means an exhaustive list - if you're looking for something more specific, it's fairly easy to find it with a simple online search!\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## Working With Open Data\n",
    "\n",
    "See our series of open data examples!\n",
    "\n",
    "1. [Car mileage data part 1](Importing%20and%20working%20with%20open%20data.ipynb)\n",
    "1. [Car mileage data part 2](Importing%20and%20Working%20With%20Open%20Data%20Part%202.ipynb)\n",
    "1. [Lotto 649 Data](Lottery.ipynb)\n",
    "1. [Meteorite Landings Part 1](Meteorite%20Landings.ipynb)\n",
    "1. [Meteorite Landings Part 2](Meteorite%20Landings%20Part%202.ipynb)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](https://github.com/callysto/callysto-sample-notebooks/blob/master/notebooks/images/Callysto_Notebook-Banners_Bottom_06.06.18.jpg?raw=true)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
